One-Stage Tree: end-to-end tree builder and pruner

Abstract Decision trees have favorable properties, including interpretability, high computational efficiency, and the ability to learn from little training data. Learning a decision tree is known be NP-complete. The researchers proposed many greedy algorithms such as CART approximate solutions. Inspired by current popular neural networks, soft that support end-to-end with back-propagation attracted more attention. However, existing either lose interpretability due continuous relaxation or employ two-stage method of building then pruning. In this paper, we propose One-Stage Tree build prune jointly through bilevel optimization problem. Moreover, leverage reparameterization trick proximal iterations keep discrete during training. As result, reduces performance gap between testing maintains advantage interpretability. Extensive experiments demonstrate outperforms on classification regression tasks.